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Abstract 

Two losing games, when alternated in a periodic or random fashion, 
can produce a winning game. This paradox occurs in a family of stochastic 
processes: if one combines two or more dynamics where a given quantity 
decreases, the result can be a dynamic system where this quantity in- 
creases. The paradox could be applied to a number of stochastic systems 
and has drawn the attention of researchers from different areas. In this 
paper we show how the phenomenon can be used to design Brownian or 
molecular motors, i.e., thermal engines that operate by rectifying fluctua- 
tions. We briefly review the literature on Brownian motors, pointing out 
that a new thermodynamics of Brownian motors will be fundamental to 
the understanding of most processes of energy transduction in molecular 
biology. 



1 Paradoxical games 

Suppose we have a biased coin, so that the probability that any flip will result 
in a head is 0.5 — e, where e is a small and positive number. With this coin, call 
it coin 1, it is proposed that you play the following game: if the flip results in a 



tail you receive 1 dollar, if not you pay the same amount. If you have unlimited 
capital then you should accept to play the game without hesitation: if x(t) is 
your opponent's capital after t runs, it will not be difficult for you to show that 
its mean value, (x(t)) is a strictly a decreasing function of t (while your average 
capital is an increasing function of t). 

Now, it is proposed that you play a second game that uses two coins, call 
them coin 2 and coin 3 respectively. Coin 3 is flipped whenever your opponent's 
capital is a multiple of 3, in any other case coin 2 is flipped (note that your 
opponent can have negative capital; by a multiple of 3 it is understood any 
integer number that can be written as 3rt, with n an integer number). Let 
P2 = 0.75 — e and p$ = 0.1 — e be the probabilities that your opponent wins with 
coin 2 and coin 3 respectively, see Figure The analysis of this game is not as 
simple as that of our previous game. However, it can be shown - see Appendix 
1 and Figure^™ that this game, too, is favourable for you in the sense that the 
mean value of the opponent's capital (x(t)} is, once again, a strictly decreasing 
function of t. Let us call the first game we have described "game A" and the 
second one "game B." 

Once you are convinced that your opponent will lose in both games you 
are given a third proposal: alternate the games following the sequence AAB- 
BAABB... If you frown, the proposal can be modified to make it less suspicious: 
in each run we will randomly chose the game that is played. If you accept either 
of these proposals you would have trusted your intuition too much, not realizing 
that random systems, even as simple as the ones we have just described, may 
behave in an unexpected way. 



Game A 



Gome B 




Is Xfi) a multiple of 3? 



Noj ^Yes 

Coin 2 Coin 3 



Win Lose 



Win Lose 



Figure 1: Rules of games A and B. The values of the probabilities are: p\ — \ — e, 
P2 = j — e and p% = jq — e, with e small and positive, which in all cases favour 
the "losing" option. The lighter colour of coin 2 indicates that with this coin 
the probability of winning is greater than | and so it is a favourable coin. 
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Games played 

Figure 2: Average capital (over 50 000 players), as a function of the number of 
runs, for each of the games and in several combinations, including a random 
combination. In all cases e = 0.005 and we have used the notation [a, b] to 
indicate that we play a runs of game A, followed by b runs of game B, and so 
on. The random combination does not differ much from the combination [2, 2], 
that is, the combinaton AABBAA...( After [T]). 

Indeed, if we alternate the games, either by following a fixed sequence AAB- 
BAABB... or chosing randomly the game we play, then your opponent's average 
capital (x(t)) will be a strictly increasing function of t. Figure [3 shows your op- 
ponent's capital in different situations: games A and B played alone, in several 
periodic combinations and in a random combination. 

2 Detailed analysis of the games 

The phenomenon we have just described is known as Parrondo's paradox and it 
is receiving some attention ^ 2] because it is thought it might have application 
in areas such as economics and evolutionary theory. It is true that this phe- 
nomenon may arise in any situation that involves the interplay of two or more 
random dynamics. However, we have not yet described a real situation where 
the paradox takes place. 

Let us see how we can analyse these paradoxical games. Game B and the 
random combination of games A and B may be reduced to a 3-state Markov 
chain These states are: the capital is multiple of 3, multiple of 3 plus 1 or 
multiple of 3 plus 2. The variable that determines these states is 
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y = x mod 3 (1) 

which only takes on the values 0, 1 and 2. The analysis of this Markov chain 
can be carried out by diagonalizing a 3x3 matrix (see Appendix 1). 

This analysis gives us the following intuitive explanation of the paradox. 
Game B uses two coins: one "bad", coin 3, and one "good", coin 2. When game 
B is played alone, the probability of using coin 3 is 

5 440 

7Tq = e + ... (2) 

13 2197 v ' 

where we have dropped the terms of order e 2 to simplify the analysis. Notice, 
also, that the probability ttq is not equal to i, as one might have initially 
thought, but greater than |. Consequently, the probability of winning is 

1 147 

Pwin = 7T P3 + (1 - 7T0)P2 = ~ - — 6 + . . . (3) 

which is less than i for any positive e. Nevertheless, when games A and B 
are combined randomly, the probability of using coin 3 becomes 

, _ 245 48880 
*° ~ 709 ~ 50268i e + ' ' ' { ) 

which is less than ttq- The probability of winning is now 

p / , Ps+Pi , n ,s P2+Pi 727 486795 

F.,,:„ = 7T n h ( 1 — 7T n ) = e + . . . ( 5 ) 

2 V 0> 2 1418 502681 V ' 

which is greater than ~ if e is small enough. What happens then is that game 
A, even though it consists of just one "bad" coin, redistributes the frequencies 
of use of the two coins of game B in such a way that the "good" coin is being 
used more often than its counterpart. This is the essence of the paradox: the 
winning tendency is already built-in into game B, but when this game is played 
alone, the losing tendency is dominant. The role of game A is to reverse this 
dominancy. Thus, game A, in spite of being a losing game, effects a reprise of 
the "good" coin of game B that overcomes its own losing tendency, so that the 
combination of games A and B becomes a winning game. 



3 A simple Brownian motor 

What has all this to do with physics? Even though now the paradox is part 
of probability theory, it was originally inspired by a physical system, namely, a 
Brownian motor or molecular motor [5] . 

With a slight change in the winning and losing probabilities, the games 
described above could be interpreted as a Brownian particle in one-dimension 
subjected to certain potentials. 

Let us consider the following model of a Brownian particle in one-dimension, 
where the particle can occupy the discrete set of positions x = 0, ±1±2, . . .. The 
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particle jumps with certain probability from one position to another at discrete 
times t = 0, 1, 2, ... Suppose the particle is under the action of a certain periodic 
potential V(x) of period 3, i.e. V(x + 3) = V(x). This means the potential is 
completely defined by the three values: V(0), V(l) and V(2), which we denote 
by Vo, V\ and V%, respectively. We will set Vq = 0, Vi = V and V2 = -j, so 
that the potential is asymmetric (see Figure |3J). In addition, suppose there is 
an external force F acting on the particle and pointing to the left. The energy 
at each point is then 

E(x) = V(x) + Fx. (6) 

How does this particle move at a certain temperature T? When a physical 
system is at a certain temperature T and can be found in different states i with 
energy Ei its evolution may be described by a Markov chain |H] . The dynamics 
are expressed in terms of the transition probabilities p[i —* j) for jumping from 
state i to the state j. These probabilities must satisfy the detailed-balance 
condition. One of the dynamics to satisfy this requirement is the Metropolis 
algorithm (see Appendix 2) that is used in Monte-Carlo simulations of systems 
in equilibrium at a certain temperature T 




Figure 3: The potential used on the Brownian particle described in this article. 

When our Brownian particle evolves following the Metropolis algorithm at 
a finite temperature T, its position x(t) behaves more or less like the capital 
in game B. On the other hand, if we follow the Metropolis algorithm at a very 
high temperature, the resulting system is very much like game A. In both cases 
the external force plays the role of the games' e parameter. These analogies give 
us a strong indication how to reproduce the paradox in the behaviour of the 
Brownian particle. At any temperature the particle moves to the left because 
this is the direction of the external force F. Hence, the mean value of (x(t)) 
decreases with t. On the other hand, if we alternate two different temperatures 
T\ and T2, the particle moves to the right against the external force, that is (x(t)) 
increases with t. Again, two dynamics where (x(t)) decreases give rise, when 
combined, to dynamics where (x(t)) increases. This phenomenon is depicted 
in Figure there we plot the position of the particle as a function of time, 
obtained by numerical simulation for the temperatures, T\ = 1 and T2 = 0.3, 
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and the alternating sequence TiTiT 2 T 2 TiTi. 

Even more interesting is to check that the particle is a thermal engine. Let 
us consider a periodic alternating sequence of the type T{T{TiTiT{T\ . . ., that 
is, we put the particle in contact with a thermal bath at temperature T\ for two 
time-steps, then in contact with a thermal bath at temperature T 2 for the next 
two time-steps, and so on. Let us assume T\ > T 2 . We have, thus, a system 
in contact with two thermal baths at different temperatures just as in the well 
known cycles of the classical thermal engines. So, the particle behaves exactly 
as a thermal engine, that is, it extracts energy from the hot source, does work 
against the external force and dissipates part of the extracted energy in the form 
of heat towards the cold source. 




Figure 4: The position of the Brownian particle for the two temperatures, T\ = 1 
and T-2 — 0.3 and the alternating sequence T\T\TiTiT\T\. Just like in the games 
(Figure El we clearly see the inversion of [the direction of] movement when we 
alternate the two temperatures. 

Let us see how to quantitatively study the energy exchange of this engine. 
In a certain step t — ► t + 1, in which the particle is in contact with, say, bath 1, 
the energy transferred from the bath to the particle is 

oo 

Q&t+i = £ M* + 1) - (*)] (V(x) + Fx) (7) 

x— — oo 

where ir x (t) is the probability that the particle is at the position x in time t. 
The right hand side of the above equation can be decomposed into two terms 

Q t ( H t+1 = AU t ^ t+1 - WWhi. (8) 

The first term is AUt^t+i = Mt+i — Wf, where U(t) is the internal energy of 
the system 
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oo 

U{t) = Mt)V(x) = (V(x(t))}. (9) 

X— — OO 

On the other hand, the second term is the change of energy due to the 
external force F, that is, the work done by the force on the particle 

WUt+i = -F(v(t)) (10) 

where (v) = (x(t + 1)) — (x(t)) is the mean velocity of the particle. Hence 
Equation (|10|l has a simple interpretation: the work done by the force on the 
particle in a unit of time, or the power developed by the force, is the product of 
the force times the mean velocity of the particle. The minus sign arises because 
the direction of the force points to the left. 




FORCE 

Figure 5: The efficiency r\ of the thermal engine (continuous line), work W done 
in each cycle (circles) and the current of particles (dotted line) as a function of 
the external force F for T\ =1,T 2 = 0.1 and V = 1. 

The three quantities we have defined, the internal energy, heat and work 
are related through Equation JSJ, which is nothing else that the First Law of 
Thermodynamics 1 (the sign convention we follow is the usual one in physics, 
namely, that heat and work are both positive if there is energy entering the 
system) . 

The same procedure is applied to the time-steps when the system is in con- 
tact with bath 2. When we alternate the two baths in a cycle of four steps, 
T1T1T2T2 . . ., the system, after a certain number of cycles, reaches a regime 
where the internal energy and the mean velocity are periodic in time. In this 

An alternative viewpoint considers the term —Fx(t) as part of the internal energy. In 
this interpretation, the heat dissipated in each bath is the same as the one computed in this 
article. There are changes, however, in the work, which is always zero, and the internal energy. 
This interpretation has two problems. Firstly, the engine is not strictly cyclic, because the 
internal energy increases in each cycle. And secondly, it is not possible to define in a simple 
way the efficiency of the engine. 
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regime we can compute the amount of heat extracted from each bath and the 
total work done by the force. The analytical treatment is analogous to the one 
carried out for the games and it is based on a three-state Markov chain. We 
shall not include here the details of the calculations but shall give some of the 
results that can be obtained. Applying equation Q to the first two steps of the 
cycle we compute the energy Q\ transferred from bath 1 to the system, which is 
positive if T± > T<i. However, when we apply the same equation to the last two 
steps of the cycle we obtain a negative energy Q2, and, if F is weak enough, we 
obtain a negative W = —Q\ — Q2- These results tell us that the system extracts 
energy from the hot bath, dissipates part of this energy towards the cold bath 
and uses what remains of it to do work against the external force. The efficiency 
of the thermal engine is 

which is positive and less than 1. 

In Figure we plot the engine efficiency as a function of the external force 
F. When the force is zero the efficiency is zero because the engine does not do 
any work and there is an irreversible heat transfer from the hot source to the 
cold one. Note that this holds even though in this case there exists a current 
of particles. For sufficiently strong forces the efficiency is once again zero. The 
reason is that the particles cannot move against strong forces. On the other 
hand, there exists a force for which the particle is, on the average, at rest and 
does not do work, while there still exists heat transfer from the hot bath to the 
cold one. 




Figure 6: Efficiency of the thermal engine as a funtion of the temperature of 
the cold source T2, for T-y = 1 and several values of V. For each point, we have 
taken the value of the external force that maximizes the efficiency 



As it can be seen, the engine always works irreversibly, so it can never attain 
the efficiency of the Carnot engine. This is a common fact in the majority of 
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Brownian motors |SJ \7\ (an exception are the reversible ratchets |Hj , whose effi- 
ciencies can be arbitrarily close to Carnot's 0), including the Feynman ratchet 
9 . It is worth pointing out here that Feynman did not notice the intrinsic 
irreversibility of his thermal engine and went on to show that under a certain 
hypothesis that the engine could attain the Carnot efficiency. It has recently 
been shown that such hypotheses are incorrect |1U|. 

In Figure we show the engine's efficiency for several temperatures T 2 
keeping fixed the temperature T\. Observe that the Carnot efficiency curve, 
1 — ^ = I — T 2 , is well above all the curves plotted in Figure El 

4 Conclusions 

We have shown a physical application of the paradoxical games, namely, in 
Brownian or molecular motors. The name "molecular motor" comes from bio- 
physics where several forms of energy conversion within the cell are studied. 
For example, there are "pumps" capable of transporting ions from one side of 
a biological membrane to the other against the ion's electrochemical gradient 
^2E]i molecules travelling through microtubules transporting diverse cellu- 
lar material; or molecules capable of "pulling off' a filament and making up 
the muscular tissue ^2J- Each one of these systems consists of a unique protein 
that changes its tri-dimensional configuration upon hydrolysis of ATP molecules. 
The protein is different for each system considered (kinesins in the transport 
through micro-channels, myosines in the muscular tissue), but all of them are 
motors capable of extracting the stored energy in the ATP bonds and transform 
it into mechanical energy. All these transformations are carried out at a scale 
where the thermal fluctuations are unavoidable. 

For all this, we believe that in order to understand the energy-exchange 
processes at the cellular level a disposal of the authentic thermodynamics of 
Brownian motors will be essential. This thermodynamics will have to be, as we 
have seen, one of irreversible processes wherein the thermal fluctuations play 
a relevant role. It will, also, have to be different from the so-called finite-time 
thermodynamics, since the latter studies irreversible motors but of macroscopic 
character. 

In recent years, the first steps have been taken towards building up the 
theory of molecular motors. Nevertheless, despite some work that shows general 
properties [3 El El E] most of the results so far refer to concrete models 
0] |S] that are too simple to represent the behaviour of molecular motors. 

Finally, we would like to conclude by recalling that the paradoxical games 
may have applications in other contexts. The paradox shows that the result of 
alternating two stochastic dynamics is far from being the "sum" or combina- 
tion of the effects of each dynamic considered alone, and that could, in fact, 
be completely unexpected. This result should raise the interest in alternating 
dynamics in physical, biological and economics systems. 
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Appendix 1 — Markov Chain 



A Markov chain is a set of N states and we consider a particle that jumps in 
from one state to another in a probabilistic way and in discrete time steps. The 
Markov chain is defined by the matrix 

n = {j^}£. =1 (12) 

where p(i — > j) is the particle transition probability from state i to the state 
j. Contrary to the usual matrix notation we shall use i as the column index 
and j as the row index. It can be readily checked that all the matrix elements 
are real numbers between and 1 and that the entries of every column add up 
to 1. 

Let us call 7Tj(i) the probability that the particle is in the state i in time t. 
The probability to find the particle in the state i at time t + 1 is equal to the 
probability that the particle jumps, in the step t — > t + 1, from any other state 
j (including the case i = j), that is 

N 

TTi(t+l)=J2^(t)Pj^i- (13) 

Hence the distribution of probability n(t) = (TTi(t), . . . 7T/v(t)) satisfies the 
following evolution equation 

7?(* + 1) = Il7f(i). (14) 

The distribution ir(t) tends, at large t, to a stationary distribution, 7rf, that 
satisfies equation (|15fl 

ir st (t+l) = Il7r st (i) (15) 

that is, 7r st is the eigenvector of the matrix II with eigenvalue 1. 

The stationary distribution tells us, in the limit of long times, what are the 
probabilities of the particle being in the different states i = 1, 2, ... N. 

Game B consists of a Markov chain of three states, namely, 0, 1 and 2. The 
matrix II is 

(0 1 - p 2 p2 \ 
P3 l-pa . 

1 - P3 P2 / 

Notice that the elements of every column add up to 1 
probability distribution is 

7f st = — (l P2P3 + 1 ~ P2 P2P3 + 1 ~PA 
2\' l-P2+P 2 2 ' l-P2+P 2 2 J 

where Z is a normalization constant. Finally, a similar analysis holds for the 
random combination of games A and B upon substituting p 2 for ( p2 + Pl ) and 



(16) 

The stationary 
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P3 for i£i±£ii. From this stationary probability distribution 7r|* it is easy to 
compute the probability of winning in a particular run, as we have done in the 
main text. 



Appendix 2 — The Metropolis algorithm 

Let us suppose we have a physical system that can be in any of the states 
i = 1, 2, ... TV with energy E^. What are the transition probabilities p(i — * j) 
that govern the probabilistic evolution of the system when this is in contact 
with with a thermal bath at temperature T? 

The requirement these transition probabilities have to satisfy is that they 
should eventually attain the Boltzmann distribution, that is, its stationary dis- 
tribution should be 

jfrt = L e -PE,/kT (18) 

where Z is a normalized constant and (3 = tip, with k the Boltzmann con- 
stant. Consequently, the transition matrix II has to be such that it has the 
distribution 7r st , given by l|18|). as its eigenvector with eigenvalue 1. 

A particular case is when the transition probabilities satisfy the condition of 
detailed balance 

nf Pl ^ = Trfp^i. (19) 

An even more particular case is the Metropolis algorithm wherein the tran- 
sition probabilities are given by the following rule, for i =/= j: or i and j are not 
connected, in which case p(i — > j) = p(J — ► i) = 0, or they are connected, so 
that 

f TT ^ — 

= \ ^ e -(^-Si)/3 if Ei > E h (20) 

where Ci is chosen so that all the transition probabilities are between and 
1 (it often is the coordination number, i.e. the number of states connected to 
the state i) Finally, the transition probability from i to itself is 

Pi-i = 1 -^Pi-y (21) 

where the sum is carried over all the states j connected to i. It can readily 
be checked that the transition probabilities so defined satisfy the condition of 
detailed balance. 

We shall use the Metropolis algorithm in this article to model a particle 
in contact with a thermal bath. Notice also that in Equation H20|) there is a 
non-zero probability that the particle increases its energy. These transitions are 
the result of thermal fluctuations where the thermal bath gives energy to the 
particle. 
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